SemanticScuttle - klotz.me

Tags: vram* + quantization* + context length* + cli* + llm*

0 bookmark(s) - Sort by: Date ↓ / Title /

1. [Script] Calculate VRAM requirements for LLM models

   A ruby script calculates VRAM requirements for large language models (LLMs) based on model, bits per weight, and context length. It can determine required VRAM, maximum context length, or best bpw given available VRAM.

   2024-08-01 Tags: llm, vram, script, quantization, model size, context length, cli, linux, ruby reddit, foss by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0